Method and apparatus for decoding/encoding a video signal with inter-view reference picture list construction

ABSTRACT

A method of decoding a video signal is disclosed. The present invention includes obtaining identification information indicating whether a coded picture of a current NAL unit is an inter-view picture group, obtaining interview reference information of the inter-view picture group, and constructing a reference picture list for inter-view prediction using the inter-view reference information according to the identification information, wherein the inter-view reference information includes view identification information of an inter-view reference picture.

This application is a continuation of U.S. application Ser. No.12/310,103 filed Feb. 11, 2009, which claims priority from PCTInternational Application No. PCT/KR2007/004115 filed on Aug. 27, 2009,which claims priority on U.S. provisional application Nos. 60/840,032;60/842,152; 60/842,151; and 60/869,867 filed on Aug. 25, 2006, Sep. 5,2006, Sep. 5, 2006 and Dec. 13, 2006. The entire contents of all ofthese applications are incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to video signal coding.

BACKGROUND ART

Compression coding means a series of signal processing techniques fortransmitting digitalized information via a communication circuit orstoring the digitalized information in a form suitable for a storagemedium. As targets of compression coding, there are audio, video,characters, etc. In particular, a technique for performing compressioncoding on video is called video sequence compression. A video sequenceis generally characterized in having spatial redundancy or temporalredundancy.

DISCLOSURE OF THE INVENTION Technical Problem

Accordingly, the present invention is directed to a method and apparatusfor decoding/encoding a video signal that can substantially enhanceefficiency in coding the video signal.

Technical Solution

Accordingly, the present invention is directed to a method and apparatusfor decoding/encoding a video signal that substantially obviate one ormore of the problems due to limitations and disadvantages of the relatedart.

An object of the present invention is to perform coding on a videosignal efficiently by defining view information capable of identifying aview of picture.

Another object of the present invention is to efficiently perform codingon a video signal by defining an inter-view prediction flag indicatingwhether a coded picture of a current NAL unit is used for inter-viewprediction.

Another object of the present invention is to efficiently perform codingon a video, signal by constructing and managing a reference picture listusing inter-view reference information indication an inter-viewdependent relation.

Another object of the present invention is to efficiently perform aninter-view random access by obtaining inter-view reference informationbased on inter-view picture group identification information.

Another object of the present invention is to efficiently perform codingon a video signal by providing a method of managing reference picturesused for inter-view prediction.

Another object of the present invention is to efficiently perform codingon a video signal by providing a method of constructing a referencepicture list for inter-view prediction.

A further object of the present invention is to efficiently performcoding on a video signal by providing a method of modifying a referencepicture list for inter-view prediction.

Advantageous Effects

The present invention provides the following effects or advantages.

First of all, in coding a video signal, the present invention provides amethod of managing reference pictures used for inter-view prediction,thereby enabling efficient coding.

Secondly, the present invention provides a method of initializing areference picture list for inter-view prediction and a method ofmodifying a reference picture list for inter-view prediction, therebyenabling efficient coding.

Thirdly, in case that inter-view prediction is carried out by thepresent invention, a burden of a DPB (decoded picture buffer) isreduced. So, a coding rate can be enhanced and more accurate predictionis enabled to reduce the number of bits to be transported.

Fourthly, more efficient coding is enabled using various kinds ofconfiguration informations on a multi-view sequence.

DESCRIPTION OF DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this specification, illustrate embodiments of the invention andtogether with the description serve to explain the principles of theinvention.

In the drawings:

FIG. 1 is a schematic block diagram of a video signal decoding apparatusaccording to the present invention;

FIG. 2 is a diagram of configuration informations on a multi-viewsequence that can be added to a multi-view sequence coded bit streamaccording to one embodiment of the present invention;

FIG. 3 is a block diagram of a reference picture list constructing unit620 according to one embodiment of the present invention;

FIG. 4 is a diagram of an overall prediction structure of a multi-viewsequence signal according to one embodiment of the present invention toexplain a concept of an inter-view picture group;

FIG. 5 is a flowchart for constructing and modifying a reference picturelist according to one embodiment of the present invention;

FIG. 6 is a diagram for a method of constructing a reference picturelist when a current slice is a slice-P according to one embodiment ofthe present invention;

FIG. 7 is a diagram for a method of constructing a reference picturelist when a current slice is a slice-B according to one embodiment ofthe present invention;

FIG. 8 is a block diagram of a reference picture list modifying unit 640according to one embodiment of the present invention;

FIG. 9 is a block diagram of a reference number assignment changing unit643B/645B according to one embodiment of the present invention; and

FIG. 10 is a diagram for a process for modifying a reference picturelist using view information according to one embodiment of the presentinvention.

BEST MODE

Additional features and advantages of the invention will be set forth inthe description which follows, and in part will be apparent from thedescription, or may be learned by practice of the invention. Theobjectives and other advantages of the invention will be realized andattained by the structure particularly pointed out in the writtendescription and claims thereof as well as the appended drawings.

To achieve these and other advantages and in accordance with the purposeof the present invention, as embodied and broadly described, a method ofdecoding a video signal according to the present invention includesobtaining identification information indicating whether a coded pictureof a current NAL unit is an inter-view picture group, obtaininginter-view reference information of the inter-view picture group, andconstructing a reference picture list for inter-view prediction usingthe inter-view reference information according to the identificationinformation, wherein the inter-view reference information includes viewidentification information of an inter-view reference picture.

In a method of decoding a video signal including obtainingidentification information indicating whether a coded picture of acurrent NAL unit is an inter-view picture group, obtaining inter-viewreference information of the inter-view picture group, and constructinga reference picture list for inter-view prediction using the inter-viewreference information according to the identification information,wherein the inter-view reference information includes viewidentification information of an inter-view reference picture, the videosignal is characterized in being received as a broadcast signal.

In a method of decoding a video signal including obtainingidentification information indicating whether a coded picture of acurrent NAL unit is an inter-view picture group, obtaining inter-viewreference information of the inter-view picture group, and constructinga reference picture list for inter-view prediction using the inter-viewreference information according to the identification information,wherein the inter-view reference information includes viewidentification information of an inter-view reference picture, the videosignal is characterized in being received via a digital medium.

To further achieve these and other advantages and in accordance with thepurpose of the present invention, a medium, in which a program forexecuting a method of decoding a video signal including obtainingidentification information indicating whether a coded picture of acurrent NAL unit is an inter-view picture group, obtaining inter-viewreference information of the inter-view picture group, and constructinga reference picture list for inter-view prediction using the inter-viewreference information according to the identification information,wherein the inter-view reference information includes viewidentification information of an inter-view reference picture isrecorded, is configured to be read by a computer.

To further achieve these and other advantages and in accordance with thepurpose of the present invention, an apparatus for decoding a videosignal includes a variable deriving unit obtaining identificationinformation indicating whether a coded picture of a current NAL unit isan inter-view picture group and inter-view reference information of theinter-view picture group and a reference picture list constructing unitconstructing a reference picture list for inter-view prediction usingthe inter-view reference information according to the identificationinformation, wherein the inter-view reference information includes viewidentification information of an inter-view reference picture.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and areintended to provide further explanation of the invention as claimed.

MODE FOR INVENTION

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings.

First of all, compression coding of video signal data considers spatialredundancy, spatial redundancy, scalable redundancy, and inter-viewredundancy. And, compression coding is enabled by considering inter-viewexisting mutual redundancy in the course of the compression coding.Compression coding scheme, which takes inter-view redundancy intoconsideration, is just an embodiment of the present invention. And, thetechnical idea of the present invention is applicable to temporalredundancy, scalable redundancy, and the like.

In the present disclosure, coding can include both concepts of encodingand decoding. And, coding can be flexibly interpreted to correspond tothe technical idea and scope of the present invention.

Looking into a configuration of a bit stream in H.264/AVC, there existsa separate layer structure called a NAL (network abstraction layer)between a VCL (video coding layer) dealing with a moving pictureencoding process itself and a lower system that transports and storesencoded information. An output from an encoding process is VCL data andis mapped by NAL unit prior to transport or storage. Each NAL unitincludes compressed video data or RBSP (raw byte sequence payload:result data of moving picture compression) that is the datacorresponding to header information.

The NAL unit basically includes a NAL header and an RBSP. The NAL headerincludes flag information (nal_ref_idc) indicating whether a slice as areference picture of the NAL unit is included and an identifier(nal_unit_type) indicating a type of the NAL unit. Compressed originaldata is stored in the RBSP. And, RBSP trailing bit is added to a lastportion of the RBSP to represent a length of the RBSP as an 8-bitmultiplication. As the type of the NAL unit, there is IDR (instantaneousdecoding refresh) picture, SPS (sequence parameter set), PPS (pictureparameter set), SEI (supplemental enhancement information), or the like.

In the standardization, restrictions for various profiles and levels areset to enable implementation of a target product with an appropriatecost. In this case, a decoder should meet the restriction decidedaccording the corresponding profile and level. Thus, two concepts,‘profile’ and ‘level’ are defined to indicate a function or parameterfor representing how far the decoder can cope with a range of acompressed sequence. And, a profile indicator (profile_idc) can identifythat a bit stream is based on a prescribed profile. The profileindicator means a flag indicating a profile on which a bit stream isbased. For instance, in H.264/AVC, if a profile indicator is 66, itmeans that a bit stream is based on a baseline profile. If a profileindicator is 77, it means that a bit stream is based on a main profile.If a profile indicator is 88, it means that a bit stream is based on anextended profile. And, the profile identifier can be included in asequence parameter set.

So, in order to deal with a multi-view video, it needs to be identifiedwhether a profile of an inputted bit stream is a multi-view profile. Ifthe profile of the inputted bit stream is the multi-view profile, it isnecessary to add syntax to enable at least one additional informationfor multi-view to be transmitted. In this case, the multi-view profileindicates a profile mode handling multi-view video as an amendmenttechnique of H.264/AVC. In MVC, it may be more efficient to add syntaxas additional information for an MVC mode rather than unconditionalsyntax. For instance, when a profile indicator of AVC indicates amulti-view profile, if information for a multi-view video is added, itis able to enhance encoding efficiency.

A sequence parameter set indicates header information containinginformation crossing over coding of an overall sequence such as aprofile, a level, and the like. A whole compressed moving picture, i.e.,a sequence should begin at a sequence header. So, a sequence parameterset corresponding to header information should arrive at a decoderbefore data referring to the parameter set arrives. Namely, the sequenceparameter set RBSP plays a role as the header information for the resultdata of the moving picture compression. Once a bit stream is inputted, aprofile indicator preferentially identifies that the inputted bit streamis based on which one of a plurality of profiled. So, by adding a partfor deciding whether an inputted bit stream relates to a multi-viewprofile (e.g., ‘If (profile_idc==MULTI_VIEW_PROFILE)’) to syntax, it isdecided whether the inputted bit stream relates to the multi-viewprofile. Various kinds of configuration information can be added only ifthe inputted bit stream is approved as relating to the multi-viewprofile.

For instance, it is able to add a number of total views, a number ofinter-view reference pictures, a view identification number of aninter-view reference picture, and the like. And, a decoded picturebuffer can use various kinds of informations on an inter-view referencepicture to construct and manage a reference picture list. This will beexplained in detail with reference to FIGS. 5 to 10.

FIG. 1 is a schematic block diagram of an apparatus for decoding a videosignal according to the present invention.

Referring to FIG. 1, an apparatus for decoding a video signal accordingto the present invention includes a NAL parser 100, an entropy decodingunit 200, an inverse quantization/inverse transform unit 300, anintra-prediction unit 400, a deblocking filter unit 500, a decodedpicture buffer unit 600, an inter-prediction unit 700, and the like.

The decoded picture buffer unit 600 includes a reference picture storingunit 610, a reference picture list constructing unit 620, a referencepicture managing unit 650, and the like. And, the reference picture listconstructing unit 620 includes a variable deriving unit 625, a referencepicture list initializing unit 630, and a reference picture listmodifying unit 640.

The NAL parser 100 carries out parsing by NAL unit to decode a receivedvideo sequence. In general, at least one sequence parameter set and atleast one picture parameter set are transferred to a decoder before aslice header and slice data are decoded. In this case, various kinds ofconfiguration informations can be included in a NAL header area or anextension area of a NAL header. Since MVC is an amendment technique fora conventional AVC technique, it may be more efficient to add theconfiguration informations in case of an MVC bit stream only rather thanunconditional addition. For instance, it is able to add flag informationfor identifying a presence or non-presence of an MVC bit stream in theNAL header area or the extension area of the NAL header. Only if aninputted bit stream is a multi-view video coded bit stream according tothe flag information, it is able to add configuration informations for amulti-view video.

For instance, the configuration informations can include viewidentification information, inter-view picture group identificationinformation, inter-view prediction flag information, temporal levelinformation, priority identification information, identificationinformation indicating whether it is an instantaneous decoded picturefor a view, and the like. They are explained in detail with reference toFIG. 2.

FIG. 2 is a diagram of configuration informations for a multi-viewsequence addable to a multi-view sequence coded bit stream according toone embodiment of the present invention.

FIG. 2 shows an example of a NAL-unit structure to which configurationinformations for a multi-view sequence can be added. NAL unit can mainlyinclude NAL unit header and RBSP (raw byte sequence payload: result dataof moving picture compression).

The NAL unit header can include identification information (nal_ref_idc)indicating whether the NAL unit includes a slice of a reference pictureand information (nal_unit_type) indicating a type of the NAL unit.

And, an extension area of the NAL unit header can be conditionallyincluded.

For instance, if the information indicating the type of the NAL unit isassociated with scalable video coding or indicates a prefix NAL unit,the NAL unit is able to include an extension area of the NAL unitheader. In particular, if the nal_unit_type=20 or 14, the NAL unit isable to include the extension area of the NAL unit header. And,configuration informations for a multi-view sequence can be added to theextension area of the NAL unit header according to flag information(svc_mvc_flag) capable of identifying whether it is MVC bit stream.

For another instance, if the information indicating the type of the NALunit is information indicating a sequence parameter set, the RBSP caninclude information on the sequence parameter set. In particular, ifnal_unit_type=7, the RBSP can include information for a sequenceparameter set. In this case, the sequence parameter set can include anextension area of the sequence parameter set according to profileinformation. For example, if profile information (profile_idc) is aprofile relevant to multi-view video coding, the sequence parameter setcan include an extension area of the sequence parameter set.Alternatively, a subset sequence parameter set can include an extensionarea of a sequence parameter set according to profile information. Theextension area of the sequence parameter set can include inter-viewreference information indicating inter-view dependency.

Various configuration informations on a multi-view sequence, e.g.,configuration informations that can be included in an extension area ofNAL unit header or configuration informations that can be included in anextension area of a sequence parameter set are explained in detail asfollows.

First of all, view identification information means information fordistinguishing a picture in a current view from a picture in a differentview. In coding a video sequence signal, POC (picture order count) and‘frame_num’ are used to identify each picture. In case of a multi-viewvideo sequence, inter-view prediction is carried out. So, identificationinformation to discriminate a picture in a current view from a picturein another view is needed. Thus, it is necessary to define viewidentification information for identifying a view of a picture. The viewidentification information can be obtained from a header area of a videosignal. For instance, the header area can be a NAL header area, anextension area of a NAL header, or a slice header area. Information on apicture in a view different from that of a current picture is obtainedusing the view identification information and it is able to decode thevideo signal using the information on the picture in the different view.

The view identification information is applicable to an overallencoding/decoding process of the video signal. For instance, viewidentification information can be used to indicate inter-viewdependency. Number information of reference picture, view identificationinformation of an inter-view reference picture and the like may beneeded to indicate the inter-view dependency. Like the numberinformation of the inter-view reference picture and the viewidentification information of the inter-view reference picture,informations used to indicate the inter-view dependency are calledinter-view reference information. In this case, the view identificationinformation can be used to indicate the view identification informationof the inter-view reference picture. The inter-view reference picturemay mean a reference picture used in performing inter-view prediction ona current picture. And, the view identification information can beintactly applied to multi-view video coding using ‘frame_num’ thatconsiders a view instead of considering a specific view identifier.

Inter-view picture group identification information means informationcapable of identifying whether a coded picture of a current NAL unit isan inter-view picture group. In this case, the inter-view picture groupmeans a coded picture in which all slices reference only slices with thesame picture order count. For instance, it means a coded picture thatrefers to a slice in a different view only but does not refer to a slicein a current view. In decoding a multi-view sequence, an inter-viewrandom access is possible. For inter-view prediction, inter-viewreference information is necessary. In obtaining the inter-viewreference information, inter-view picture group identificationinformation is usable. For instance, if a current picture corresponds toan inter-view picture group, inter-view reference information on theinter-view picture group can be obtained. If a current picturecorresponds to a non-inter-view picture group, inter-view referenceinformation on the non-inter-view picture group can be obtained. Thiswill be explained in detail with reference to FIGS. 5 to 10.

Thus, in case that inter-view reference information is obtained based oninter-view picture group identification information, it is able toperform inter-view random access more efficiently. This is becauseinter-view reference relation between pictures in an inter-view picturegroup can differ from that in a non-inter-view picture group. And, incase of an inter-view picture group, pictures in a plurality of viewscan be referred to. For instance, a picture of a virtual view isgenerated from pictures in a plurality of views and it is then able topredict a current picture using the picture of the virtual view.

In constructing a reference picture list, the inter-view picture groupidentification information can be used. In this case, the referencepicture list can include a reference picture list for inter-viewprediction. And, the reference picture list for the inter-viewprediction can be added to the reference picture list. For instance, incase of initializing a reference picture list or modifying the referencepicture list, the inter-view picture group identification informationcan be used. And, it can be also used to manage the added referencepictures for the inter-view prediction. For instance, by dividing thereference pictures into an inter-view picture group and a non-inter-viewpicture group, it is able to mark a representation indicating thatreference pictures failing to be used in performing inter-viewprediction shall not be used. And, the inter-view picture groupidentification information is applicable to a hypothetical referencedecoder. This will be explained in detail with reference to FIGS. 5 to10.

Inter-view prediction flag information means information indicatingwhether a coded picture of a current NAL unit is used for inter-viewprediction. The inter-view prediction flag information is usable for apart where temporal prediction or inter-view prediction is performed. Inthis case, identification information indicating whether NAL unitincludes a slice of a reference picture can be used together. Forinstance, although a current NAL unit fails to include a slice of areference picture according to the identification information, if it isused for inter-view prediction, the current NAL unit can be a referencepicture used for only an inter-view prediction. According to theidentification information, if a current NAL unit includes a slice of areference picture and used for inter-view prediction, the current NALunit can be used for temporal prediction and inter-view prediction.Although NAL unit fails to include a slice of a reference pictureaccording to the identification information, it can be stored in adecoded picture buffer. This is because, in case that a coded picture ofa current NAL unit is used for inter-view prediction according to theinter-view prediction flag information, it needs to be stored.

Aside from a case of using both of the flag information and theidentification information together, one identification information canindicate whether a coded picture of a current NAL unit is used fortemporal prediction or/and inter-view prediction.

And, the inter-view prediction information can be used for a single loopdecoding process. In case that a coded picture of a current NAL unit isnot used for inter-view prediction according to the inter-viewprediction flag information, decoding can be performed in part. Forinstance, intra-macroblock is completely decoded, whereasinter-macroblock is partially decoded for only a residual information.Hence, it is able to reduce complexity of a decoder. This can beefficient if it is unnecessary to reconstruct a sequence by specificallyperforming motion compensation in different views when a user is lookingat a sequence in a specific view only without viewing a sequence in allviews.

The diagram shown in FIG. 4 is used to explain one embodiment of thepresent invention.

For instance, a coding order may correspond to S0, S2 and S1 inconsidering a portion of the diagram shown in FIG. 4. Let's assume thata picture to be currently coded is a picture B₃ on a time zone T2 in aview S1. In this case, a picture B₂ on the time zone T2 in a view S0 anda picture B₂ on the time zone T2 in a view S2 can be used for inter-viewprediction. If the picture B₂ on the time zone T2 in the view S0 is usedfor the inter-view prediction, the inter-view prediction flaginformation can be set to 1. If the picture B₂ on the time zone T2 inthe view S0 is not used for the inter-view prediction, the flaginformation can be set to 0. In this case, if inter-view prediction flaginformation of all slices in the view S0 is 0, it may be unnecessary todecode the all slices in the view S0. Hence, coding efficiency can beenhanced.

For another instance, if inter-view prediction flag information of allslices in the view S0 is not 0, i.e., if at least one slice is set to 1,a slice set to 0 should be decoded even if the slice is set to 0. Sincethe picture B₂ on the time zone T2 in the view S0 is not used fordecoding of a current picture, assuming that decoding is not executed bysetting the inter-view prediction information to 0, it is unable toreconstruct a picture B₃ on the time zone T1 in the view S0, whichrefers to the picture B₂ on the time zone T2 in the view S0, and apicture B₃ on a time zone T3 in the view S0 in case of decoding slicesin the view S0. Hence, slices set to 0 should be reconstructedregardless of the inter-view prediction flag information.

For further instance, the inter-view prediction flag information isusable for a decoded picture buffer (DPB). For example, if theinter-view prediction flag information is not provided, the picture B₂on the time zone T2 in the view S0 should be unconditionally stored inthe decoded picture buffer. Yet, if it is able to know that theinter-view prediction flag information is 0, the picture B₂ on the timezone T2 in the view S0 may not be stored in the decoded picture buffer.Hence, it is able to save a memory of the decoded picture buffer.

Temporal level information means information on a hierarchical structureto provide temporal scalability from a video signal. Though the temporallevel information, it is able to provide a user with a sequence onvarious time zones.

Priority identification information means information capable ofidentifying a priority of NAL unit. It is able to provide viewscalability using the priority identification information. For examples,it is able to define view level information using the priorityidentification information. In this case, view level information meansinformation on a hierarchical structure for providing view scalabilityfrom a video signal.

In a multi-view video sequence, it is necessary to define a level for atime and a level for a view to provide a user with various temporal andview sequences. In case of defining the above level information, it isable to use temporal scalability and view scalability. Hence, a user isable to view a sequence at a specific time and view only or a sequenceaccording to another condition for restriction only.

The level information can be set in various ways according to itsreferential condition. For instance, the level information can be setdifferent according to camera location or camera alignment. And, thelevel information can be determined by considering inter-view referenceinformation. For instance, a level for a view of which an inter-viewpicture group is an I picture is set to 0, a level for a view of whichan inter-view picture group is a P picture is set to 1, and a level fora view of which an inter-view picture group is a B picture is set to 2.Thus, the level value can be assigned to the priority identificationinformation. Moreover, the level information can be arbitrarily set notbased on a special reference.

Meanwhile, the entropy decoding unit 200 carries out entropy decoding ona parsed bit stream, and a coefficient of each macroblock, a motionvector, and the like are then extracted. The inversequantization/inverse transform unit 300 obtains a transformedcoefficient value by multiplying a received quantized value by aconstant and then transforms the coefficient value inversely toreconstruct a pixel value. Using the reconstructed pixel value, theintra-prediction unit 400 performs an intra prediction from a decodedsample within a current picture. Meanwhile, the deblocking filter unit500 is applied to each coded macroblock to reduce block distortion. Afilter smoothens a block edge to enhance an image quality of a decodedframe. A selection of a filtering process depends on boundary strengthand gradient of an image sample around a boundary. Pictures throughfiltering are outputted or stored in the decoded picture buffer unit 600to be used as reference pictures.

The decoded picture buffer unit 600 plays a role in storing or openingthe previously coded pictures to perform an inter prediction. In thiscase, to store the pictures in the decoded picture buffer unit 600 or toopen the pictures, ‘frame_num’ and POC (picture order count) of eachpicture are used. So, since there exist pictures in a view differentfrom that of a current picture among the previously coded pictures, viewinformation for identifying a view of a picture may be usable togetherwith the ‘frame_num’ and the POC. The decoded picture buffer unit 600includes the reference picture storing unit 610, the reference picturelist constructing unit 620, and the reference picture managing unit 650.The reference picture storing unit 610 stores pictures that will bereferred to for the coding of the current picture. The reference picturelist constructing unit 620 constructs a list of reference pictures forthe inter-picture prediction. In multi-view video coding, inter-viewprediction may be needed. So, if a current picture refers to a picturein another view, it may be necessary to construct a reference picturelist for the inter-view prediction.

Moreover, it is able to construct a reference picture list forperforming both temporal prediction and inter-view prediction. Forinstance, if a current picture refers to a picture in a diagonaldirection, it is able to construct a reference picture list in thediagonal direction. In this case, there are various methods forconstructing the reference picture list in the diagonal direction. Forexample, it is able to define information (ref_list_idc) for identifyinga reference picture list. If ref_list_idc=0, it means a referencepicture list for temporal prediction. If it is 1, it indicates areference picture list for inter-view prediction. If it is 2, it canindicate a reference picture list for both temporal prediction andinter-view prediction.

The reference picture list in the diagonal direction can be constructedusing the reference picture list for the temporal prediction or thereference picture list for the inter-view prediction. For instance, itis able to add reference pictures in a diagonal direction to a referencepicture list for temporal prediction. Alternatively, it is able to addreference pictures in a diagonal direction to a reference picture listfor inter-view prediction. Like this, if lists in various directions areconstructed, more efficient coding is possible. In the presentdisclosure, the reference picture list for the temporal prediction andthe reference picture list for the inter-view prediction are mainlydescribed. And, the concept of the present invention is applicable to areference picture list in a diagonal direction as well.

The reference picture list constructing unit 620 can use viewinformation in constructing the reference picture list for theinter-view prediction. For instance, inter-view reference informationcan be used. Inter-view reference information means information used toindicate an inter-view dependency. For instance, there can be a numberof total views, a view identification number, a number of inter-viewreference pictures, a view identification number of an inter-viewreference picture, and the like. This will be explained in detail withreference to FIG. 4.

FIG. 3 is an internal block diagram of a reference picture listconstructing unit 620 according to an embodiment of the presentinvention.

The reference picture list constructing unit 620 includes the variablederiving unit 625, the reference picture list initializing unit 630, andthe reference list modifying unit 640.

The variable deriving unit 625 derives variables used for referencepicture list initialization. For instance, the variable can be derivedusing ‘frame_num’ indicating a picture identification number. Inparticular, variables FrameNum and FrameNumWrap may be usable for eachshort-term reference picture. First of all, the variable FrameNum isequal to a value of a syntax element frame_num. The variableFrameNumWrap can be used for the decoded picture buffer unit 600 toassign a small number to each reference picture. And, the variableFrameNumWrap can be derived from the variable FrameNum. So, it is ableto derive a variable PicNum using the derived variable FrameNumWrap. Inthis case, the variable PicNum can mean an identification number of apicture used by the decoded picture buffer unit 600. In case ofindicating a long-term reference picture, a variable LongTermPicNum canbe usable.

In order to construct a reference picture list for inter-viewprediction, it is able to derive a first variable (e.g., ViewNum) toconstruct a reference picture list for inter-view prediction using theinter-view reference information. For instance, it is able to derive asecond variable (e.g., ViewId) using ‘view_id’ for identifying a view ofa picture. First of all, the second variable can be equal to a value ofthe syntax element ‘view_id’. And, a third variable (e.g., ViewIdWrap)can be used for the decoded picture buffer unit 600 to assign a smallview identification number to each reference picture and can be derivedfrom the second variable. In this case, the first variable ViewNum canmean a view identification number of inter-view reference picture usedby the decoded picture buffer unit 600. Yet, since a number of referencepictures used for inter-view prediction in multi-view video coding maybe relatively smaller than that used for temporal prediction, it may notdefine another variable to indicate a view identification number of along-term reference picture.

The reference picture list initializing unit 630 initializes a referencepicture list using the above-mentioned variables. In this case, aninitialization process for the reference picture list may differaccording to a slice type. For instance, in case of decoding a P-slice,it is able, to assign a reference index based on a decoding order. Incase of decoding a B-slice, it is able to assign a reference index basedon a picture output-order. In case of initializing a reference picturelist for inter-view prediction, it is able to assign an index to areference picture based on the first variable, i.e., the variablederived from view identification information of inter-view referencepicture. In this case, the reference picture list can be constructed ina manner of discriminating an inter-view reference picture group andnon-inter-view reference picture group from each other.

The reference picture list modifying unit 640 plays a role in enhancinga compression efficiency by assigning a smaller index to a picturefrequently referred to in the initialized reference picture list. Thisis because a small bit is assigned if a reference index for encodinggets smaller. In this case, in order to modify a reference picture listfor inter-view prediction, inter-view reference information can be used.For instance, number information of inter-view reference picture can beused in the course of modifying the reference picture list. This will beexplained in detail with reference to FIG. 5, and FIG. 8˜FIG. 10.

And, the reference picture list modifying unit 640 includes a slice typechecking unit 642, a reference picture list-0 modifying unit 643, and areference picture list-1 modifying unit 645. If an initialized referencepicture list is inputted, the slice type checking unit 642 checks a typeof a slice to be decoded and then decides whether to modify a referencepicture list-0 or a reference picture list-1. So, the reference picturelist-0/1 reordering unit 643,645 modifies of the reference picturelist-0 if the slice type is not an I-slice and also modifies of thereference picture list-1 additionally if the slice type is a B-slice.Thus, after an end of the modifying process, a reference picture list isconstructed.

The reference picture list0/1 modifying unit 643, 645 includes anidentification information obtaining unit 643A,645A and a referenceindex assignment changing unit 643B,645B respectively. Theidentification information obtaining unit 643A,645A receivedidentification information (reordering_of_pic_nums_idc) indicating anassigning method of a reference index if modifying of a referencepicture list is carried out according to flag information indicatingwhether to modify the reference picture list. And, the reference indexassignment changing unit 643B,645B modifies the reference picture listby changing an assignment of a reference index according to theidentification information.

And, the reference picture list modifying unit 640 is operable byanother method. For instance, modifying can be executed by checking aNAL unit type transferred prior to passing through the slice typechecking unit 642 and then classifying the NAL unit type into a case ofMVC NAL and a case of non-MVC NAL.

The reference picture managing unit 640 manages reference pictures toexecute inter prediction more flexibly. For instance, a memorymanagement control operation method and a sliding window method areusable. This is to manage a reference picture memory and a non-referencepicture memory by unifying the memories into one memory and realize anefficient memory management with a small memory. In multi-view videocoding, since pictures in a view direction have the same picture ordercount, information for identifying a view of each of the pictures isusable in marking the pictures in a view direction. And, referencepictures managed in the above manner can be used by the inter-predictionunit 700.

The inter-prediction unit 700 carries out inter prediction usingreference pictures stored in the decoded picture buffer unit 600. Aninter-coded macroblock can be divided into macroblock partitions. And,each of the macroblock partitions can be predicted from one or tworeference pictures. The inter-prediction unit 700 compensates for amotion of a current block using informations transferred from theentropy decoding unit 200. Motion vectors of neighboring blocks of thecurrent block are extracted from a video signal, and then a motionvector predictor of the current block are derived from the motionvectors of the neighboring blocks. And, the motion of the current blockis compensated using the derived motion vector predictor and adifferential motion vector extracted from the video signal. And, it isable to perform the motion compensation using one reference picture or aplurality of pictures. In multi-view video coding, in case that acurrent picture refers to pictures in different views, it is able toperform motion compensation using reference picture list information forthe inter-view prediction stored in the decoded picture buffer unit 600.And, it is also able to perform motion compensation using vieinformation for identifying a view of the reference picture. A directmode is an coding mode for predicting motion information of a currentblock from motion information for an encoded block. Since this method isable to save a number of bits required for coding the motioninformation, compression efficiency is enhanced. For instance, atemporal direction mode predicts motion information for a current blockusing a correlation of motion information in a temporal direction. Usinga method similar to this method, the present invention is able topredict motion information for a current block using a correlation ofmotion information in a view direction.

The inter-predicted or intra-predicted pictures through theabove-explained process are selected according to a prediction mode toreconstruct a current picture. In the following description, variousembodiments providing an efficient decoding method of a video signal areexplained.

FIG. 4 is a diagram of an overall predictive structure of a multi-viewvideo signal according to one embodiment of the present invention toexplain a concept of an inter-view picture group.

Referring to FIG. 4, T0 to T100 on a horizontal axis indicate framesaccording to time and S0 to S7 on a vertical axis indicate framesaccording to view. For instance, pictures at T0 mean frames captured bydifferent cameras on the same time zone T0, while pictures at S0 meansequences captured by a single camera on different time zones. And,arrows in the drawing indicate predictive directions and predictiveorders of the respective pictures. For instance, a picture P0 in a viewS2 on a time zone TO is a picture predicted from I0, which becomes areference picture of a picture P0 in a view S4 on the time zone T0. And,it becomes a reference picture of pictures B1 and B2 on time zones T4and T2 in the view S2, respectively.

In a multi-view video decoding process, an inter-view random access maybe needed. So, an access to a random view should be possible byminimizing the decoding effort. In this case, a concept of an inter-viewpicture group may be needed to realize an efficient access. Thedefinition of the inter-view picture group was mentioned in FIG. 2. Forinstance, in FIG. 6, if a picture I0 in a view S0 on a time zone T0 isan inter-view picture group, all pictures in different views on the sametime zone, i.e., the time zone T0, become inter-view picture groups. Foranother instance, if a picture I0 in a view S0 on a time zone T8 is aninter-view picture group, all pictures in different views on the sametime zone, i.e., the time zone T8, are inter-view picture groups.Likewise, all pictures in T16, . . . , T96, and T100 become inter-viewpicture groups as well.

In an overall predictive structure of MVC, GOP can begin with anI-picture. And, the I-picture is compatible with H.264/AVC. So, allinter-view picture groups compatible with H.264/AVC can always becomethe I-picture. Yet, in case that the I-pictures are replaced by aP-picture, more efficient coding is enabled. In particular, moreefficient coding is enabled using the predictive structure enabling GOPto begin with the P-picture compatible with H.264/AVC.

In this case, if the inter-view picture group is re-defined, all slicesbecome encoded picture capable of referring to not only a slice in aframe on a same time zone but also a slice in the same view on adifferent time zone. Yet, in case of referring to a slice on a differenttime zone in a same view, it can be restricted to the inter-view picturegroup compatible with H.264/AVC only. After the inter-view picture grouphas been decoded, all of the sequentially coded pictures are decodedfrom pictures decoded ahead of the inter-view picture group in an outputorder without inter-prediction.

Considering the overall coding structure of the multi-view video shownin FIG. 4, since inter-view reference information of an inter-viewpicture group differs from that of a non-inter-view picture group, it isnecessary to distinguish the inter-view picture group and thenon-inter-view picture group from each other according to the inter-viewpicture group identification information.

The inter-view reference information means the information capable ofrecognizing a predictive structure between inter-view pictures. This canbe obtained from a data area of a video signal. For instance, it can beobtained from a sequence parameter set area. And, the inter-viewreference information can be recognized using the number of referencepictures and view information for the reference pictures. For instance,the number of total views is obtained and the view information foridentifying each view can be then obtained based on the number of thetotal views.

And, number information for interview reference pictures, whichindicates a number of reference pictures for a reference direction ofeach view can be obtained. According to the number information of theinter-view reference pictures, it is able to obtain view identificationinformation of each inter-view reference picture. Through this method,the inter-view reference information can be obtained. And, theinter-view reference information can be obtained in a manner of beingcategorized into a case of an inter-view picture group and a case of anon-inter-view picture group. This can be known using inter-view picturegroup identification information indicating whether a coded slice in acurrent NAL corresponds to an inter-view picture group. The inter-viewpicture group identification information can be obtained from anextension area of NAL header, or a slice layer.

The inter-view reference information obtained according to theinter-view picture group identification information is usable forconstruction and modification of a reference picture list. This will beexplained in detail with reference to FIGS. 5 to 10.

FIG. 5 is a flowchart for constructing and modifying a reference picturelist according to one embodiment of the present invention.

Flag information indicating coding scheme can be obtained from a headerof a received video signal. For instance, the flag informationindicating the coding scheme may include flag information indicatingscalable video coding scheme or multi-view video coding scheme. The flaginformation can be obtained from a NAL header or an extension area ofNAL header. In case that a bit stream is a multi-view video coded bitstream according to the flag information, configuration informations formulti-view video coding can be obtained from a header. For instance,view identification information, inter-view picture group identificationinformation, inter-view prediction flag information and the like can beobtained (S510). In this case, the configuration informations can beobtained from a NAL header or an extension area of NAL header.

Profile information related to multi-view video coding can be obtained(S520). For instance, the profile information can be obtained from asequence parameter set. According to the profile information indicatingthe multi-view video coding, an extension area of the sequence parameterset for the multi-view video coding can exist. Inter-view referenceinformation can be obtained from the extension area of the sequenceparameter set. In this case, the inter-view reference information can beobtained based on the inter-view picture group identificationinformation and the inter-view reference information can include viewidentification information and number information of the inter-viewreference picture (S530).

A reference picture list for inter-view prediction can be constructedusing the view identification information of the inter-view referencepicture (S540).

The reference picture list can be modified using number information ofthe inter-view reference picture (S550).

The modified reference picture list for the inter-view prediction can bemanaged by the reference picture managing unit 640 to realizeinter-picture prediction more flexibly (S560). For this, an adaptivememory management control operation method and a sliding window methodare usable for example.

Detailed embodiments for how a reference picture list is constructed andmodified according to a slice type are explained with reference to FIG.6 and FIG. 7 as follows.

FIG. 6 is a diagram explain a method of constructing a reference picturelist when a current slice is a P-slice according to one embodiment ofthe present invention.

Referring to FIG. 6, a time is indicated by T0, T1, . . . , TN, while aview is indicated by V0, V1, . . . , V4. For instance, a current pictureindicates a picture at a time T3 in a view V4. And, a slice type of thecurrent picture is a P-slice. ‘PN’ is an abbreviation of a variablePicNum, ‘LPN’ is an abbreviation of a variable LongTermPicNum, and ‘VN’is an abbreviation of a variable ViewNum. A numeral attached to an endportion of each of the variables indicates an index indicating a time ofeach picture (for PN or LPN) or a view of each picture (for VN). This isapplicable to FIG. 7 in the same manner.

A reference picture list for temporal prediction or a reference picturelist for inter-view prediction can be generated in a different wayaccording to a slice type of a current slice. For instance, a slice typein FIG. 6 is a P/SP slice. In this case, a reference picture list 0 isgenerated. In particular, the reference picture list 0 can include areference picture list for temporal prediction and/or a referencepicture list for inter-view prediction. In the present embodiment, it isassumed that a reference picture list includes both a reference picturelist for temporal prediction and a reference picture list for inter-viewprediction.

There are various methods for ordering reference pictures. For instance,reference pictures can be aligned according to in order of decoding orpicture output. Alternatively, reference pictures can be aligned basedon a variable derived using view information. Alternatively, referencepictures can be aligned according to inter-view reference informationindicating an inter-view prediction structure.

In case of a reference picture list for temporal prediction, short-termreference pictures and long-term reference pictures can be aligned basedon a decoding order. For instance, they can be aligned according to avalue of a variable PicNum or LongTermPicNum derived from a valueindicating a picture identification number (e.g., frame_num orLongtermframeidx). First of all, short-term reference pictures can beinitialized prior to long-term reference pictures. An order of aligningthe short-term reference pictures can be set from a reference picturehaving a highest value of variable PicNum to a reference picture havinga lowest variable value. For instance, the short-term reference picturescan be aligned in order of PN1 having a highest variable, PN2 having anintermediate variable, and PNO having a lowest variable among PN0 toPN2. An order of aligning the long-term reference pictures can be setfrom a reference picture having a lowest value of variableLongTermPicNum to a reference picture having a highest variable value.For instance, the long-term reference pictures can be aligned in orderof LPN0 having a highest variable and LPN1 having a lowest variable.

In case of a reference picture list for inter-view prediction, referencepictures can be aligned based on a first variable ViewNum derived usingview information. In this case, the view information can be inter-viewreference information. The inter-view reference information can includeview information of an inter-view reference picture. And, the firstvariable can be derived from the view information of the inter-viewreference picture. In particular, reference pictures can be aligned inorder of a reference picture having a highest first variable (ViewNum)value to a reference picture having a lowest first variable (ViewNum)value. For instance, in FIG. 6, reference pictures can be aligned inorder of VN3 having a highest variable, VN2, VN1, and VN0 having alowest variable among VN0, VN1, VN2, and VN3.

Thus, both of the reference picture list for the temporal prediction andthe reference picture list for the inter-view prediction can be managedas one reference picture list. Alternatively, both of the referencepicture list for the temporal prediction and the reference picture listfor the inter-view prediction can be managed as separate referencepicture lists, respectively. In case of managing both of the referencepicture list for the temporal prediction and the reference picture listfor the inter-view prediction as one reference picture list, they can beinitialized according to an predetermined order or simultaneously. Forinstance, in case of initializing both of the reference picture list forthe temporal prediction and the reference picture list for theinter-view prediction according to an determined order, the referencepicture list for the temporal prediction is preferentially initializedand the reference picture list for the inter-view prediction is theninitialized in addition.

According to another embodiment, an inter-view reference picture listcan be constructed according to inter-view picture group identificationinformation. And, an inter-view reference picture list can beconstructed based on an inter-view prediction flag. In this case,construction can include the meaning of initialization. For instance, ifa current picture corresponds to an inter-view picture group accordingto inter-view picture group identification information, an inter-viewreference picture list can be constructed using view identificationinformation of an inter-view reference picture corresponding to theinter-view picture group. If a current picture corresponds to anon-inter-view picture group, view identification information of aninter-view reference picture corresponding to the non-inter-view picturegroup is usable. In this case, the inter-view reference picture or thenon-inter-view reference picture can have the same value of information(e.g., picture order count value) indicating a picture output order ofthe current picture. The inter-view reference picture is usable forinter-view prediction according to an inter-view prediction flag. Forinstance, if the inter-view prediction flag is 1, a slice of a currentNAL can be used the inter-view prediction. So, in order to construct aninter-view reference picture list using the inter-view referencepicture, the inter-view prediction flag for the inter-view referencepicture should be set to 1.

According to another embodiment, if a current picture corresponds to aninter-view picture group, a temporal reference picture list may not beconstructed. In this case, an inter-view reference picture list can beconstructed only. This is because inter-view prediction can existbetween inter-view picture groups. So, if an inter-view picture group isdefined in a different manner, even if a current picture is aninter-view picture group, a temporal reference picture list can beconstructed. This is because temporal prediction as well as inter-viewprediction can exist between non-inter-view picture groups.

The above-explained embodiments are applicable to FIG. 7 as well. A casethat a slice type of a current picture is a B-slice is explained withreference to FIG. 7 as follows.

FIG. 7 is a diagram to explain a method constructing a reference picturelist when a current slice is a B-slice according to one embodiment ofthe present invention.

Referring to FIG. 7, in case that a slice type is a B-slice, a referencepicture list 0 and a reference picture list 1 are generated. In thiscase, the reference picture list 0 or the reference picture list 1 caninclude a reference picture list for temporal prediction only or both areference picture list for temporal prediction and a reference picturelist for inter-view prediction. Alternatively, a reference picture listfor inter-view prediction can be included only.

In case of the reference picture list for the temporal prediction, ashort-term reference picture aligning method may differ from a long-termreference picture aligning method. For instance, in case of short-termreference pictures, reference pictures can be aligned according to apicture order count. In case of long-term reference pictures, referencepictures can be aligned according to a variable (LongtermPicNum) value.And, the short-term reference pictures can be initialized prior to thelong-term reference pictures.

In order of aligning short-term reference pictures of the referencepicture list 0, reference pictures are preferentially aligned from areference picture having a highest POC value to a reference picturehaving a lowest POC value among reference pictures having POC valuessmaller than that of a current picture, and then aligned from areference picture having a lowest POC value to a reference picturehaving a highest POC value among reference pictures having POC valuesgreater than that of the current picture. For instance, referencepictures can be preferentially aligned from PN1 having a highest POCvalue in reference pictures PN0 and PN1 having. POC values smaller thanthat of a current picture to PN0, and then aligned from PN3 having alowest POC value in reference pictures PN3 and PN4 having a POC valuesmaller than that of a current picture to PN4.

In order of aligning long-term reference pictures of the referencepicture list 0, reference pictures are aligned from a reference picturehaving a lowest variable LongtermPicNum to a reference picture having ahighest variable. For instance, reference pictures are aligned from LPN0having a lowest value in LPN0 and LPN1 to LPN1 having a second lowestvariable.

In case of the reference picture list for the inter-view prediction,reference pictures can be aligned based on a first variable ViewNumderived using view information. In this case, the view information canbe inter-view reference information. The inter-view referenceinformation can include view information of an inter-view referencepicture. And, the first variable can be derived from the viewinformation of the inter-view reference picture.

For instance, in case of the reference picture list 0 for the inter-viewprediction, reference pictures can be aligned from a reference picturehaving a highest first variable value among reference pictures havingfirst variable values lower than that of a current picture to areference picture having a lowest first variable value. The referencepictures are then aligned from a reference picture having a lowest firstvariable value among reference pictures having first variable valuesgreater than that of the current picture to a reference picture having ahighest first variable value. For instance, reference pictures arepreferentially aligned from VN1 having a highest first variable value inVN0 and VN1 having first variable values smaller than that of a currentpicture to VN0 having a lowest first variable value and then alignedfrom VN3 having a lowest first variable value in VN3 and VN4 havingfirst variable values greater than that of the current picture to VN4having a highest first variable value.

In case of the reference picture list 1, the above-explained aligningmethod of the reference list 0 is similarly applicable.

First of all, in case of the reference picture list for the temporalprediction, in order of aligning short-term reference pictures of thereference picture list 1, reference pictures are preferentially alignedfrom a reference picture having a lowest POC value to a referencepicture having a highest POC value among reference pictures having POCvalues greater than that of a current picture and then aligned from areference picture having a highest POC value to a reference picturehaving a lowest POC value among reference pictures having POC valuessmaller than that of the current picture. For instance, referencepictures can be preferentially aligned from PN3 having a lowest POCvalue in reference pictures PN3 and PN4 having POC values greater thanthat of a current picture to PN4 and then aligned from PN1 having ahighest POC value in reference pictures PN0 and PN1 having POC valuesgreater than that of the current picture to PN0.

In order of aligning long-term reference pictures of the referencepicture list 1, reference pictures are aligned from a reference picturehaving a lowest variable LongtermPicNum to a reference picture having ahighest variable. For instance, reference pictures are aligned from LPN0having a lowest value in LPN0 and LPN1 to LPN1 having a lowest variable.

In case of the reference picture list for the inter-view prediction,reference pictures can be aligned based on a first variable ViewNumderived using view information. In this case, the view information canbe inter-view reference information. The inter-view referenceinformation can include view information of an inter-view referencepicture. And, the first variable can be derived from the viewinformation of the inter-view reference picture.

For instance, in case of the reference picture list 1 for the inter-viewprediction, reference pictures can be aligned from a reference picturehaving a lowest first variable value among reference pictures havingfirst variable values greater than that of a current picture to areference picture having a highest first variable value. The referencepictures are then aligned from a reference picture having a highestfirst variable value among reference pictures having first variablevalues smaller than that of the current picture to a reference picturehaving a lowest first variable value. For instance, reference picturesare preferentially aligned from VN3 having a lowest first variable valuein VN3 and VN4 having first variable values greater than that of acurrent picture to VN4 having a highest first variable value and thenaligned from VN1 having a highest first variable value in VN0 and VN1having first variable values smaller than that of the current picture toVN0 having a lowest first variable value.

The reference picture list initialized by the above process istransferred to the reference picture list modifying unit 640. Theinitialized reference picture list is then reordered for more efficientcoding. The modifying process is to reduce a bit rate by assigning asmall number to a reference picture having highest probability in beingselected as a reference picture by operating a decoded picture buffer.Various methods of modifying a reference picture list are explained withreference to FIGS. 8 to 10 as follows.

FIG. 8 is an internal block diagram of the reference picture listmodifying unit 640 according to one embodiment of the present invention.

Referring to FIG. 8, the reference picture list modifying unit 640basically includes a slice type checking unit 642, a reference picturelist 0 modifying unit 643, and a reference picture list 1 modifying unit645.

In particular, the reference picture list 0 modifying unit 643 includesa first identification information obtaining unit 643A, and a firstreference index assignment changing unit 643B. And, the referencepicture list 1 modifying unit 645 includes a second identificationobtaining unit 645A and a second reference index assignment changingunit 645B.

The slice type checking unit 642 checks a slice type of a current slice.It is then decided whether to modify reference picture list 0 and/or areference picture list 1 according to the slice type. For instance, if aslice type of a current slice is an I-slice, both of the referencepicture list 0 and the reference picture list 1 are not modified. If aslice type of a current slice is a P-slice, the reference picture list 0is modified only. If a slice type of a current slice is a B-slice, bothof the reference picture list 0 and the reference picture list 1 aremodified.

The reference picture list 0 modifying unit 643 is activated if flaginformation for modifying the reference picture list 0 is ‘true’ and ifthe slice type of the current slice is not the I-slice. The firstidentification information obtaining unit 643A obtains identificationinformation indicating a reference index assigning method. The firstreference index assignment changing unit 643B changes a reference indexassigned to each reference picture of the reference picture list 0according to the identification information.

Likewise, the reference picture list 1 modifying unit 645 is activatedif flag information for modifying the reference picture list 1 is ‘true’and if the slice type of the current slice is the B-slice. The secondidentification information obtaining unit 645A obtains identificationinformation indicating a reference index assigning method. The secondreference index assignment changing unit 645B changes a reference indexassigned to each reference picture of the reference picture list 1according to the identification information.

So, reference picture list information used for actual inter-predictionis generated through the reference picture list 0 modifying unit 643 andthe reference picture list 1 modifying unit 645.

A method of changing a reference index assigned to each referencepicture by the first or second reference index assignment changing unit643B or 645B is explained with reference to FIG. 9 as follows.

FIG. 9 is an internal block diagram of a reference index assignmentchanging unit 643B or 645B according to one embodiment of the presentinvention. In the following description, the reference picture list 0modifying unit 643 and the reference picture list 1 modifying unit 645shown in FIG. 8 are explained together.

Referring to FIG. 9, each of the first and second reference indexassignment changing units 643B and 645B includes a reference indexassignment changing unit for temporal prediction 644A, a reference indexassignment changing unit for long-term reference picture 644B, areference index assignment changing unit for inter-view prediction 644C,and a reference index assignment change terminating unit 644D. Accordingto identification informations obtained by the first and secondidentification information obtaining units 643A and 645A, parts withinthe first and second reference index assignment changing units 643B and645B are activated, respectively. And, the modifying process keeps beingexecuted until identification information for terminating the referenceindex assignment change is inputted.

For instance, if identification information for changing assignment of areference index for temporal prediction is received from the first orsecond identification information obtaining unit 643A or 645A, thereference index assignment changing unit for temporal prediction 644A isactivated. The reference index assignment changing unit for temporalprediction 644A obtains a picture number difference according to thereceived identification information. In this case, the picture numberdifference means a difference between a picture number of a currentpicture and a predicted picture number. And, the predicted picturenumber may indicate a number of a reference picture assigned rightbefore. So, it is able to change the assignment of the reference indexusing the obtained picture number difference. In this case, the picturenumber difference can be added/subtracted to/from the predicted picturenumber according to the identification information.

For another instance, if identification information for changingassignment of a reference index to a designated long-term referencepicture is received, the reference index assignment changing unit for along-term reference picture 644B is activated. The reference indexassignment changing unit for a long-term reference picture 644B obtainsa long-term reference picture number of a designated picture accordingto the identification number.

For another instance, if identification information for changingassignment of a reference index for inter-view prediction is received,the reference index assignment changing unit for inter-view prediction644C is activated. The reference index assignment changing unit forinter-view prediction 644C obtains view information difference accordingto the identification information. In this case, the view informationdifference means a difference between a view number of a current pictureand a predicted view number. And, the predicted view number may indicatea view number of a reference picture assigned right before. So, it isable to change assignment of a reference index using the obtained viewinformation difference. In this case, the view information differencecan be added/subtracted to/from the predicted view number according tothe identification information.

In, changing the reference number assignment, inter-view referenceinformation can be used. The inter-view reference information caninclude number information of inter-view reference pictures. And, it isable to derive a variable for reference picture list modification fromthe number information of the inter-view reference pictures. Forinstance, in modifying a reference picture list for temporal prediction,in case that a difference value of the view information is added orsubtracted, it is able to subtract or add the derived variable together.In this case, the derived variable can be a variable indicating thenumber information of the inter-view reference pictures. Instead, thederived variable can means a maximum value among index values indicatingviews. If a current picture corresponds to an inter-view picture groupaccording to inter-view picture group identification information, thevariable can be derived from number information of inter-view referencepictures corresponding to the inter-view picture group. If a currentpicture corresponds to a non-inter-view picture group according tointer-view picture group identification information, the variable can bederived from number information of inter-view reference picturescorresponding to the non-inter-view picture group.

The number information of the inter-view reference pictures can beobtained for each view as many as number information of total views. Incase of deriving the variable, number information of inter-viewreference pictures for a current picture can be used based on currentview information.

For another instance, if identification information for terminating areference index assignment change is received, the reference indexassignment change terminating unit 644D is activated. The referenceindex assignment change terminating unit 644D terminates an assignmentchange of a reference index according to the received identificationinformation. So, the reference picture list reordering unit 640generates reference picture list information.

Thus, reference pictures used for inter-view prediction can be managedtogether with reference pictures used for temporal prediction.Alternatively, reference pictures used for inter-view prediction can bemanaged separate from reference pictures used for temporal prediction.For this, new informations for managing the reference pictures used forthe inter-view prediction may be required. Details of the referenceindex assignment changing unit for inter-view prediction 644C areexplained with reference to FIG. 10 as follows.

FIG. 10 is a diagram to explain a process for modifying a referencepicture list using view information according to one embodiment of thepresent invention.

Referring to FIG. 10, if a view number VN of a current picture is 3, ifa size of a decoded picture buffer DPBsize is 4, and if a slice type ofa current slice is a P-slice, a modifying process for a referencepicture list 0 is explained as follows.

First of all, an initially predicted view number is ‘3’ that is the viewnumber of the current picture. And, an initial alignment of thereference picture list 0 for inter-view prediction is ‘4, 5, 6, 2’({circle around (1)}). In this case, if identification information forchanging assignment of a reference index for inter-view prediction bysubtracting a view information difference is received, ‘1’ is obtainedas the view information difference according to the receivedidentification information. A newly predicted view number (=2) iscalculated by subtracting the view information difference (=1) from thepredicted view number (=3). In particular, a first index of thereference picture list 0 for the inter-view prediction is assigned to areference picture having the view number 2. And, a picture previouslyassigned to the first index can be moved to a most rear part of thereference picture list 0. So, the modified reference picture list 0 is‘2, 5, 6, 4’ ({circle around (2)}). Subsequently, if identificationinformation for changing assignment of a reference index for inter-viewprediction by subtracting the view information difference is received,‘−2’ is obtained as the view information difference according to theidentification information. A newly predicted view number (=4) is thencalculated by subtracting the view information difference (=−2) from thepredicted view number (=2). In particular, a second index of thereference picture list 0 for the inter-view prediction is assigned to areference picture having a view number 4. Hence, the modified referencepicture list 0 is ‘2, 4, 6, 5’ ({circle around (3)}). Subsequently, ifidentification information for terminating the reference indexassignment change is received, the reference picture list 0 having themodified reference picture list 0 as an end is generated according tothe received identification information ({circle around (4)}). Hence,the order of the finally generated reference picture list 0 for theinter-view prediction is ‘2, 4, 6, 5’.

In this case, in the process of adding or subtracting the differencevalue of the view information, it is able to perform a subtraction oraddition operation on the variable value (e.g., number information ofinter-view reference pictures, maximum value among index valuesindicating views, etc.) explained in FIG. 9 according to a predeterminedconditional formula. In this case, the predetermined conditional formulamay mean a conditional formula to reduce a value, which is considerablybig in assigning an index value, into a smaller value.

For another instance of modifying the rest of the pictures after thefirst index of the reference picture list 0 for the inter-viewprediction has been assigned, a picture assigned to each index can bemoved to a position right behind that of the corresponding picture. Inparticular, a second index is assigned to a picture having a view number4, a third index is assigned to a picture (view number 5) to which thesecond index was assigned, and a fourth index is assigned to a picture(view number 6) to which the third index was assigned. Hence, thereordered reference picture list 0 becomes ‘2, 4, 5, 6’. And, asubsequent modifying process can be executed in the same manner.

The reference picture list generated by the above-explained process isused for inter-prediction. Both of the reference picture list for theinter-view prediction and the reference picture list for the temporalprediction can be managed as one reference picture list. Alternatively,each of the reference picture list for the inter-view prediction and thereference picture list for the temporal prediction can be managed as aseparate reference picture list.

As mentioned in the foregoing description, the decoding/encoding device,to which the present invention is applied, is provided to atransmitter/receiver for multimedia broadcasting such as DMB (digitalmultimedia broadcast) to be used in decoding video and data signals andthe like. And, the multimedia broadcast transmitter/receiver can includea mobile communication terminal.

A decoding/encoding method, to which the present invention is applied,is configured with a program for computer execution and then stored in acomputer-readable recording medium. And, multimedia data having a datastructure of the present invention can be stored in computer-readablerecording medium. The computer-readable recording media include allkinds of storage devices for storing data that can be read by a computersystem. The computer-readable recording media include ROM, RAM, CD-ROM,magnetic tapes, floppy discs, optical data storage devices, etc. andalso includes a device implemented with carrier waves (e.g.,transmission via internet). And, a bit stream generated by the encodingmethod is stored in a computer-readable recording medium or transmittedvia wire/wireless communication network.

INDUSTRIAL APPLICABILITY

Accordingly, while the present invention has been described andillustrated herein with reference to the preferred embodiments thereof,it will be apparent to those skilled in the art that variousmodifications and variations can be made therein without departing fromthe spirit and scope of the invention. Thus, it is intended that thepresent invention covers the modifications and variations of thisinvention that come within the scope of the appended claims and theirequivalents.

What is claimed is:
 1. A method for managing a reference picture with adecoding apparatus, comprising: receiving, with the decoding apparatus,a multi-view video data stream, the multi-view video stream includingvideo data for each of a plurality of views, each of the plurality ofviews corresponding to a different spatial angle, the multi-view videodata stream including a random access picture, the random access pictureincluding a random access slice, the random access slice referencingonly a slice corresponding to a same time and a different view of therandom access picture; initializing, with the decoding apparatus, areference picture list of the random access slice for inter-viewprediction, the initializing including ordering inter-view referencepictures for inter-view prediction in the reference picture list andassigning a reference index to each inter-view reference picture, thereference index identifying the inter-view reference picture in theinitialized reference picture list used for inter-view prediction of ablock to be decoded in the random access picture, the assigning beingperformed using inter-view reference information, the block to bedecoded having a same time with the inter-view reference pictures and adifferent view from the inter-view reference pictures, the inter-viewreference information including view number information and viewidentification information, the view number information indicating anumber of the inter-view reference pictures referred for inter-viewprediction of the random access picture, the view identificationinformation identifying to which of the plurality of views each of theinter-view reference pictures belongs, the random access picture and atleast one of the inter-view reference pictures included in theinitialized reference picture list having a same picture order count,the inter-view reference picture having inter-view predictioninformation, the inter-view prediction information indicating whether aslice is used for inter-view prediction; obtaining, with the decodingapparatus, modification information for the initialized referencepicture list from the multi-view video data stream, the modificationinformation indicating that the inter-view reference picture isreordered; obtaining, with the decoding apparatus, an assignmentmodification value for changing the reference index of the inter-viewreference picture in the initialized reference picture list according tothe modification information; and modifying, with the decodingapparatus, the initialized reference picture list of the random accessslice, the modifying including reordering the inter-view referencepicture by changing the reference index assigned to the inter-viewreference picture based on the assignment modification value, wherein inthe modifying step shifts other pictures in the initialized referencepicture list to positions later in the initialized reference picturelist.
 2. The method of claim 1, wherein the inter-view reference pictureis appended to a temporal reference picture for temporal prediction. 3.The method of claim 2, wherein the temporal reference picture fortemporal prediction and the inter-view reference picture for interviewprediction are managed as one reference picture list.
 4. The method ofclaim 1, wherein the inter-view prediction information is obtained froma Network Abstraction Layer (NAL) header extension information ofmulti-view video data.
 5. The method of claim 1, wherein the multi-viewvideo data stream includes a video data of a base view independent ofother views, the base view being a view decoded without using inter-viewprediction.
 6. The method of claim 1, wherein the reference index of theinter-view reference picture is changed by performing a subtractionoperation or an addition operation according to the modificationinformation.
 7. The method of claim 1, wherein the modificationinformation is obtained from a slice header of multi-view video data. 8.An apparatus for managing a reference picture, comprising: a parsingunit configured to receive a multi-view video data stream, themulti-view video stream including video data for each of a plurality ofviews, each of the plurality of views corresponding to a differentspatial angle, the multi-view video data stream including a randomaccess picture, the random access picture including a random accessslice, the random access slice referencing only a slice corresponding toa same time and a different view of the random access picture; areference picture list initializing unit configured to initialize areference picture list of the random access slice for inter-viewprediction, the initializing including ordering inter-view referencepictures for inter-view prediction in the reference picture list andassigning a reference index to each inter-view reference picture, thereference index identifying the inter-view reference picture in theinitialized reference picture list used for inter-view prediction of ablock to be decoded in the random access picture, the assigning beingperformed using inter-view reference information, the block to bedecoded having a same time with the inter-view reference pictures and adifferent view from the inter-view reference pictures, the inter-viewreference information including view number information and viewidentification information, the view number information indicating anumber of the inter-view reference pictures referred for inter-viewprediction of the random access picture, the view identificationinformation identifying to which of the plurality of views each of theinter-view reference pictures belongs, the random access picture and atleast one of the inter-view reference pictures included in theinitialized reference picture list having a same picture order count,the inter-view reference picture having inter-view predictioninformation, the inter-view prediction information indicating whether aslice is used for inter-view prediction; a reference picture listmodifying unit configured to obtain modification information for theinitialized reference picture list from the multi-view video datastream, the modification information indicating that the inter-viewreference picture is reordered; the reference picture list modifyingunit configured to obtain an assignment modification value for changingthe reference index of the inter-view reference picture in theinitialized reference picture list according to the modificationinformation; and the reference picture list modifying unit configured tomodify the initialized reference picture list of the random accessslice, the modifying including reordering the inter-view referencepicture by changing the reference index assigned to the inter-viewreference picture based on the assignment modification value, whereinthe modifying shifts other pictures in the initialized reference picturelist to positions later in the initialized reference picture list. 9.The apparatus of claim 8, wherein a temporal reference picture fortemporal prediction and the inter-view reference picture for interviewprediction are managed as one reference picture list.
 10. The apparatusof claim 8, wherein the inter-view prediction information is obtainedfrom a Network Abstraction Layer (NAL) header extension information ofmulti-view video data.
 11. The apparatus of claim 8, wherein themulti-view video data stream includes video data of a base viewindependent of other views, the base view being a view decoded withoutusing inter-view prediction.
 12. The apparatus of claim 8, wherein thereference index of the inter-view reference picture is changed byperforming a subtraction operation or an addition operation according tothe modification information.
 13. The apparatus of claim 8, wherein themodification information is obtained from a slice header of multi-viewvideo data.